Production-like testing and complex business to business auditing system

ABSTRACT

A system of testing updated software may include a cloud-based production environment. The system may also include a cloud-based testing environment. The system may also include a cloud-based production processing unit configured to receive in response to document, execute an initial software by inputting the document, and determine an initial software result and an initial software process based the document. The system may also include a cloud-based testing processing unit configured to receive the document, execute an updated software by inputting the document, and determine an updated software result and an updated software process based on the document. The system may also include a testing comparison unit configured to compare the initial software result and the updated software result, and compare the initial software process and the updated software process.

FIELD

The present disclosure generally relates to a data processingoptimization system. More particularly, the disclosure includes a systemand method of dynamically and scalably storing, communicating andretrieving large amounts of data.

BACKGROUND

Conventional document routing systems may not be optimized forbusiness-to-business transactions. In particular, the systems may not beable to process the volume of documents that may be applied in abusiness-to-business context. Conventional systems are not onlyexpensive, but the conventional systems also are not sufficiently fastand do not sufficiently scale.

SUMMARY

A system of testing updated software is also disclosed. In variousembodiments, the system includes a cloud-based production environment.In various embodiments, the system also includes a cloud-based testingenvironment. In various embodiments, the system also includes acloud-based production processing unit configured to receive a document,execute an initial software in the cloud-based production environment byinputting the document in the cloud-based production environment, anddetermine an initial software result and an initial software processbased the document. In various embodiments, the system also includes acloud-based testing processing unit configured to receive the document,execute an updated software in the cloud-based testing environment byinputting the document in the cloud-based testing environment, anddetermine an updated software result and an updated software processbased on the document. In various embodiments, the system also includesa testing comparison unit configured to compare the initial softwareresult and the updated software result, and compare the initial softwareprocess and the updated software process.

In various embodiments, the document is associated with a client and avendor. In various embodiments, the document comprises a plurality ofdocuments associated with the client and the vendor. In variousembodiments, the plurality of documents associated with the client andthe vendor are further associated with a time range. In variousembodiments, the document comprises a plurality of documents having adocument type. In various embodiments, a configuration of thecloud-based production environment is identical to a configuration ofthe cloud-based testing environment. In various embodiments, the systemfurther includes a display configured to display the comparison of theinitial software result and the updated software result and thecomparison of the initial software process and the updated softwareprocess. In various embodiments, the document is a document beingprocessed in a non-testing situation by an end user.

A method of testing updated software is also disclosed. In variousembodiments, the method includes constructing a cloud-based productionenvironment; constructing a cloud-based testing environment; executingan initial software in the cloud-based production environment byinputting a document; determining an initial software result and aninitial software process based on executing the initial software withthe document; executing the updated software in the cloud-based testingenvironment by inputting the document; determining an updated softwareresult and an updated software process based on executing the updatedsoftware with the document; comparing the initial software result andthe updated software result; and comparing the initial software processand the updated software process. In various embodiments, the methodfurther includes displaying, by a display, the comparison of the initialsoftware result and the updated software result and the comparison ofthe initial software process and the updated software process.

A cloud-based testing system for testing updated software is alsodisclosed. In various embodiments, the system includes a cloud-basedtesting processing unit configured to receive a document, execute anupdated software in the cloud-based testing environment by inputting thedocument in the cloud-based testing environment, and determine anupdated software result and an updated software process based on thedocument. In various embodiments, the system also includes a testingcomparison unit configured to compare the updated software result withan initial software result received from a cloud-based productionprocessing unit based on executing initial software in a cloud-basedproduction environment by inputting the document, and compare theupdated software process with an initial software process received fromthe cloud-based production processing unit based on executing theinitial software in the cloud-based production environment by inputtingthe document.

The foregoing features and elements may be combined in variouscombinations without exclusivity, unless expressly indicated otherwise.These features and elements as well as the operation thereof will becomemore apparent in light of the following description and the accompanyingdrawings. It should be understood, however, the following descriptionand drawings are intended to be exemplary in nature and non-limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present disclosure may best beobtained by referring to the detailed description and claims whenconsidered in connection with the figures, wherein like numerals denotelike elements.

FIG. 1 illustrates a block diagram of the highly scalable eventbrokering and audit traceability system, in accordance with variousembodiments;

FIG. 2 is a flowchart illustrating the process used by the highlyscalable event brokering and audit traceability system, in accordancewith various embodiments;

FIG. 3 illustrates rebalancing of listener instances based on componenthealth collections and allocation of listener hash range for eachlistener instance, in accordance with various embodiments;

FIG. 4 illustrates efficient document storage by the system, inaccordance with various embodiments;

FIG. 5 illustrates a block diagram of the multi-factor routing systemfor exchanging business transactions, in accordance with variousembodiments;

FIG. 6 is a flowchart illustrating the process used by the multi-factorrouting system for exchanging business transactions, in accordance withvarious embodiments;

FIG. 7 illustrates an exemplary architecture of the system, inaccordance with various embodiments;

FIG. 8 illustrates the system, in accordance with various embodiments;

FIG. 9 illustrates a block diagram of the system, in accordance withvarious embodiments;

FIG. 10 illustrates a process of the production-like testing and complexbusiness to business auditing system, in accordance with variousembodiments;

FIG. 11 illustrates the implementation of the system on a cloud-basedenvironment, in accordance with various embodiments; and

FIG. 12 illustrates the implementation of the system on a cloud-basedenvironment, in accordance with various embodiments.

DETAILED DESCRIPTION

The detailed description of exemplary embodiments herein makes referenceto the accompanying drawings, which show exemplary embodiments by way ofillustration. While these exemplary embodiments are described insufficient detail to enable those skilled in the art to practice theexemplary embodiments of the disclosure, it should be understood thatother embodiments may be realized and that logical changes andadaptations in design and construction may be made in accordance withthis disclosure and the teachings herein. Thus, the detailed descriptionherein is presented for purposes of illustration only and notlimitation. The steps recited in any of the method or processdescriptions may be executed in any order and are not necessarilylimited to the order presented. Furthermore, any reference to singularmay include plural embodiments, and any reference to more than onecomponent or step may include a singular embodiment or step. It is to beunderstood that reference to an item in the singular may also includethe item in the plural, and plural may include singular. All ranges andratio limits disclosed herein may be combined.

The present disclosure is described in one or more embodiments in thefollowing description with reference to the figures, in which likenumerals represent the same or similar elements. While the disclosure isdescribed in terms of the best mode for achieving the disclosure'sobjectives, it will be appreciated by those skilled in the art that itis intended to cover alternatives, modifications, and equivalents as maybe included within the spirit and scope of the disclosure as defined bythe appended claims and their equivalents as supported by the followingdisclosure and drawings.

Highly Scalable Event Brokering and Audit Traceability System

The highly scalable event brokering and audit traceability system mayinclude any supply chain collaboration system where documents areexchanged. In various embodiments, the system may be described as abusiness-to-business supply chain with business documents. The businessdocuments may include purchase orders, invoices, advance shippednotices, and/or catalogs, for example. The system may serve as afacilitator between a client and a vendor.

In various embodiments, the system is a scalable event-drivenarchitecture utilizing a single vendor to multiple clients model andproviding event history and replay. The system processes documents viaan event-driven architecture. Events represent the state of a documentas a system component processes the document. Events may also representa significant occurrence, such as an organizational change or systemnotification. All of the events associated with a document, from beingreceived by the system to a communication to a party, may be classifiedas an event flow.

A user, through a user interface, may search for an event flow based onstatus of the flow (e.g., document sent, or in-progress), based on datawithin the document itself or based on metadata about the document as aresult of a component processing the document. Once searched, the usercan see document metadata and additionally drill-down to retrieve theactual physical document.

Thus, the complete history of how a document was processed to get thedocument from one processing unit to the next is stored, easilysearchable and allows for retrying failed processing. The entire flow orany portion thereof may be restarted.

Clients may communicate business documents (e.g., purchase orders) tothe system, wherein the business documents may have one or more errors.The purchase orders may not have the correct item, the correct address,or the correct recipient, for example. In response to the purchase ordernot having the correct information, the vendor may not be able toexecute the purchase order.

For example, the purchase order may include account number dataassociated with an account number. The account number may be mapped intoa billing address or a shipping address. In many situations, clients(e.g., hospitals) have a hierarchical structure. That is, a particularclient may have a headquarters at a first location and may havefacilities at a second location. Accordingly, the ordered goods shouldbe shipped to the facilities, and not to the headquarters. Similarly,the bill or invoice should be sent to the headquarters, and not thefacility. Significant problems may arise when the first location and thesecond location are misidentified.

Conventionally, the client or the vendor may use account numbers torepresent the other party. The account numbers may be associated withthe relevant data, such as addresses. However, the client and the vendorhaving different account numbers and internal reference identifiers forproducts may create confusion. For example, if the client submits apurchase order using the client's internal reference identifier for theparticular good, the vendor may not be able to identify the particulargood, as the vendor may not be aware of which product is associated withthe client's internal reference identifier. The client's internalreference identifier may also not provide an indication of the good inany way. For example, the client may internally identify a box of latexgloves as BW36. However, the internal identifier, if accidentally used,may not be useful to the vendor if the vendor identifies the box oflatex gloves as latexgloves023. In addition, BW36 may not signify to thevendor that it is associated with latex gloves.

FIG. 1 illustrates a block diagram of an embodiment of the highlyscalable event brokering and audit traceability system. The systemincludes an engine 102, which includes an event generation unit 104, anevent detection unit 106, and an action unit 108. The system alsoincludes a display 110.

FIG. 2 illustrates a flowchart illustrating the process 200 used by thehighly scalable event brokering and audit traceability system. Theengine 102 receives, from a client, a document, such as an electronicbusiness document (e.g., a purchase order, business document invoice,advanced ship notice) (step 202). The system analyzes the document andautomatically interprets the document based on the internal referencenumbers of the client and/or the vendor. The document may be made up ofmultiple documents, and the system is configured to split up thedocuments into multiple documents. The system is configured to handlethe large amount of processing demands associated with the interpretingof the documents. The client and/or the vendor may identify to thesystem their internal identifiers for goods and other parties. In thisway, the system allows the client and the vendor to only keep track oftheir respective identifiers. In various embodiments, the client dataused to identify the client internal identifiers for goods and servicesis initially provided by the client, to the system, and is updatedperiodically. In various embodiments, the vendor data used to identifythe vendor internal identifiers for goods and services is initiallyprovided by the vendor, and is updated periodically. In variousembodiments, the client data and/or the vendor data is updated on adaily, weekly, monthly, or yearly basis, and the updating period may bedifferent for various clients and various vendors.

The system may use cloud computing to scale the facilitation ofpurchases. The system scales elastically, on-demand. That is, thevirtual computing resources will grow, and will grow until a cloudcomputing limit is reached. The virtual computing resources may growbased on a configurable setting for an algorithm provided by a vendor ofthe cloud computing resources by monitoring vital computer resources.

In response to the system receiving a document (e.g., a purchase order)from the client, the document is verified (step 202). However, thedocument may be secure, encrypted and digitally signed. As such,verifying a signature and decrypting the document may occupy significantcomputational resources. In response to the system receiving thedocument, the system replies to the client that it was received, that itis in possession of the system, and that the signature is verified. Thesignature may be verified using a de facto crypto digital signatureverification algorithm, such as Secure/Multipurpose Internet MailExtensions (S/MIME). The document is stored in a document database ofthe system and a first event, such as a structure verification event, isgenerated by the event generation unit 104 (step 204).

The event detection unit 106 detects the first event (step 206). Invarious embodiments, the first event is a signature verification event.In various embodiments, the event generation unit 104 communicates anindication to the event detection unit 106, when the first event isgenerated. In various embodiments, the event detection unit 106 monitorsoutputs of the event generation unit 104 to detect when the first eventis generated. The action unit 108 determines a first action to performbased on the detection of the first event (step 208). In variousembodiments, the action unit 108 determines a first action of verifyingthe purchase order. The action unit 108 performs the first action (step210). In various embodiments, the system verifies the structure of thedocument. Once the structure of the document is verified, another eventis generated. The structure of the document may be verified based onconfigurable metadata that defines the structure of the document oncerecognized. The next event may know when a prior event is completed byan event consumer durably marking the event as being completed. Theremay be more than one event consumer durably marking respective statuses.As one unit completes a task and hands off to another unit, a work eventis generated (step 212). The event detection unit 106 detects the secondevent (step 214) and the action unit 108 determines a second action toperform based on the second event (step 216) and the action is performedby the action unit (step 218). The system may cluster the events tosegregate the events based on time and name results for a more efficientsystem.

In various embodiments, the action unit 108 is a collection of unitsconfigured to perform individual actions, such as verification of thedocument and generation of documents. In response to the event beingcreated, it automatically triggers an action from another unit withinthe action unit 108 and the event is also recorded. The action from theanother unit may be triggered by a notification being written to durablestorage and a notification being communicated to registered eventlisteners. The event may simultaneously be a trigger for action and ahistorical data structure. Many listeners may be associated with theevent and the many listeners may be notified when the event iscompleted. Once the multiple listeners are notified, the multiplelisteners may individually perform respective processing procedures,resulting in parallel processing. The parallel processing may result insignificant efficiency gains. The system may combine the triggering andrecording results in reading and writing processing improvements as asingle command, instead of executing in two commands. In response to twocommands being processed, the database may experience bottlenecking, asunits may be communicating read and write operations simultaneously.

In addition, sharding or clustering based on the event may be executed,in order to further improve processing times. As a result, a firstengine may not fully or partially compete with another engine on thesame cluster, on the same block of events. Accordingly, the system ismore efficient and more scalable. As the scale increases, more enginesmay be elastically added and more shards may be created. The additionalengines may be added based on a configurable setting of the cloudresource provider. The additional shards may also be created based on aconfigurable setting of the cloud resource provider.

The system is highly scalable. The event broker is scalable both fromits compute layer and its persistence layer. For the compute layer, theevent broker is a client-side API able to run in the Cloud and thusscale as more compute resources are added. A central master broker maynot be included. Events may be dispatched in memory or if listeners arebusy, events may be stored to a database for other instances of thoseslow listeners to process, and thus scale. In addition to horizontalscalability, real-time configuration of client-side listeners thatdispatch settings allows for vertical scalability.

For the persistence layer, the event-broker stores events (e.g.,document metadata, organization changes, or notifications) in a shardeddatabase cluster. Each event may have a listener hash, which is used asthe shard key. Given the nature of the partition/hash, the database maybe pre-assigned shard chunks on each shard node. New shards may be addedas the event broker starts generating new events based on aconfiguration parameter with an increased listener hash range. Forexample, if the system has five shard clusters and the partition/hash is0-99, then each shard node may hold a range of 20. Adding a new shardmay simply entail expanding the partition/hash to 0-119 andpre-allocating 100-119 to the new shard. The rebalancing of listenerinstances based on component health collections and allocation oflistener hash range for each listener instance is illustrated in FIG. 3.

High-speed event processing may be achieved in a number of ways.High-speed event processing may be achieved by the listeners beingvertically scaled. Each event broker listener can have a configurablenumber of dispatcher threads/queue. High-speed event processing may beachieved by the broker being client-side and configured to behorizontally scaled. High-speed event processing may be achieved bymultiple listeners for any given event running in parallel andguaranteeing that only one instance of any specific listeners receivesthe event. High-speed event processing may be achieved by databasebatching. In database batching, database operations are batch/bulk sentto the data layer. The database operations as part of batching areincorporated together. As a result, for any given event, if there is aninsert and two updates, these can be incorporated into a single insertcommand to the database.

The system may combine three structures—the event, the command, and theaudit—and stores them. Instead of processing each event and command inthe unit section, the system prepares them in a way that they can bestored in bulk, and created in bulk. The memory is brought up so that ahigher throughput may be achieved. Increases in throughput may be on theorder of a thousand times. Searching may be performed more efficientlyas well.

Particular documents may be identified and processed at a higherpriority than other documents. Metadata structures may be used toidentify the documents with higher priority, so that they are processedfaster than others with lower priority.

By processing the three structures at the same time or similar times,larger amounts of data may be processed and computing resources may bepreserved. In various embodiments, the system is able to process 25million documents when conventional systems are unable to process even500,000 documents.

The efficient storage, the efficient retrieval, and efficient cachingprovides for the increase in processing speed. In order to achievescalability, the brokering subsystem is not tied to a master. As aresult of the clustering, every structure is independent. Accordingly,load is distributed across the various subcomponents at the lowestlevel.

Registered component listeners may have events dispatched to it eitherin-memory or via a query mechanism. An event is dispatched in-memory ifthat component is running on the same virtual machine as the eventproducer and the component listener has free thread/queue slots toaccept the event. An event may also be marked as “pending” for acomponent listener allowing an instance on another virtual machine (orthe same virtual machine) to query for the event in response to thelistener being able to process it.

In response to a listener being slow, the event bus performs querydispatch, aiding in scalability. This allows new compute resources tocome online and perform processing. In addition, an in-memory model isable to achieve high performance. Further, listener components may runall together or run in their own compute servers, resulting in veryflexible design.

Each event may be assigned a unique component listener partition or hashnumber. This partition or hash may be a simple random number 0-N. Acomponent listener may have N active listeners each running on differentservers. An event can only be processed by one and only one instance ofany given listener.

In-memory dispatches may not utilize the listener hash. However, forquery consumption, the listener hash is used to reduce contention at thedatabase level. Instances of a component listener are constantlyrebalanced so that there is always one and only one component listenerassigned to process a range of listener hashes. Listener hash alsoallows grouping or partitioning of the events to provide some minorordering or grouping guarantees in various parts of the event bus.Multiple listeners for any given event may be running in parallel whileguaranteeing that only one instance of any specific listeners receivesthe event.

For example, the system may perform 10 different sub-processes A-J toprocess the document. The sub-processes may include, for example,verifying a signature of the document, decrypting the document,identifying the format of the document, enriching the document, and/oridentifying contents and values within the document. Each of the 10different sub-processes may be associated with a unique listener, whichmay trigger execution of the respective sub-process. Therefore, each ofthe 10 different sub-processes may be executed in parallel, as eachsub-process is triggered by its respective listener. In this way, onesub-process may not wait for another sub-process to be completed, andthe system may be scaled, increasing efficiency.

Conventionally, in a common queuing system or an event sourcing system,system performance may depend on the queue, and visibility may belimited to the term of queue information. However, in the system, all ofthese things are visible to the end user, and the user can see exactlywhat's going on within the event brokering system in a way that isefficient and does not interfere with the processing phase. This isunlike other systems which may effectively be a black box. The systemevent bus allows the following visibility for any published event.

In particular, which components are going to receive the event when theyare ready to process the event, which components are actively processingthe event, and which components finished processing the event may bevisible. In addition, the event bus also provides seeing real-timeprocessing metrics, event backlog (which events are assigned but notdispatched), slow component listener (see listeners that have eventsassigned to it but the component is backlogged and is behind), andmetrics into all internal queues, threads and processing times to aid indiagnosing bottlenecks or issues.

Message brokering, which is not established for business to businesssituations, is provided by the system.

The system includes a generic event bus with the ability to send anytype of event. On top of the event bus, a document audit trackingsubsystem moves events related to transacting a document between tradingpartners into another set of document meta-data collections.

Document audit tracking provides grouping of events for a singledocument into an event flow, calculating the status of documentprocessing based on the current work performed, partial or completevisibility into each processed event (i.e., a document lifecycle), andan ability to replay events back into the event bus for re-processing bylisteners, allowing for resuming of failed document processing. Thevisibility into each processed event includes retrieving the actualdocument that is in various stages through the lifecycle from thephysical document storage engine. The document audit tracker works in aneventually-consistent paradigm, as it will eventually move data into itsdatabase collections.

As a document is processed, the document audit tracking subsystem mayrecord, in the document meta-data collections, each event that iscreated during the processing of the document. For example, when thedocument is received, a document received record may be recorded, whenthe document is verified, a document verification record may berecorded, and when the document is processed, a document process recordmay be recorded. A user of the system may be able to access a progressof the document through processing using the document audit trackingsubsystem. The system may request, from the document audit trackingsubsystem, a progress of a particular document, and the document audittracking subsystem may access the document meta-data collections todetermine a progress of the document.

In an example process, the received document may be associated with aparticular set of meta-data associated with the document, the sender,and/or the receiver. As the document is processed by the system, eachrecord recorded by the system may also be associated with the sameparticular set of meta-data, such that the document progress may betracked.

The system also includes the ability to dynamically configure and add anevent listener in order to change the workflow.

The system business to business exchanges allow monitoring of any eventand performing monitoring or alerting actions (e.g., send page, sendemail). The system business to business exchanges also allow visibilityinto dead or stalled components. The events held by these stalled ordead listeners can be released for another instance of that component topick it up and process. From the business level, documents can berestarted based on various business rules or the document processing canbe paused.

FIG. 4 illustrates efficient document storage by the system. A documentmay be very large and highly complex computing may be involved whenprocessing the document. Various system components may interact with thedocument in various ways depending on the task. To facilitate efficientaccess to and efficient use of system resources, the document may bestored in three different data formats in high availability datasystems. The document is represented in a physical document, documentprocessing metadata, and document search criteria.

Physical documents (which may entail large amounts of data) are storedin a cheap, secure storage engine that provides streaming capability.Document processing metadata may be persistently stored in a high-speedquery-able storage engine. The document meta-data contains some or allstate information about the processing of the document and providespartial or complete visibility to that processing. Document searchcriteria may be stored in a high-speed, distributed indexing engine. Adocument may be associated with hundreds of search terms.

Multi-Factor Routing System for Exchanging Business Transactions

The multi-factor routing system for exchanging business transactions mayreceive data without a sender and receiver. The system may determine thesender, receiver, protocol, and connection from which it was received.In various embodiments, the system may determine a sender and receiverbased on a connection through which the data was communicated. In manysituations, the system receives a document, such as a purchase order,with no indication of a recipient of the document. The system analyzesthe document to determine the recipient of the document (the vendor). Invarious embodiments, the vendor is identified using a vendor identifier.In various embodiments, a temporary address of the vendor may be used.In various embodiments, the document may contain nested documents, withdifferent buyers and vendors.

FIG. 5 illustrates a block diagram of the multi-factor routing systemfor exchanging business transactions. The system includes an engine 502,which includes a document identification unit 504 and a routing controlunit 506. The engine 502 is in communication with a client 508 and avendor 510.

FIG. 6 illustrates a flowchart illustrating the process 600 used by themulti-factor routing system for exchanging business transactions. Adocument identification unit 504 receives a document format associatedwith a client 508 (step 602). The document may be a purchase order orany other document. When the document is a purchase order, the documentidentification unit 504 may be a purchase order identification unit. Thedocument format may be provided by the client 508. The system does notrequire a hard-coded document structure. The system may use ametadata-driven structure, or the client may identify where to locatedata from the document, such as the sender and receiver. The locationsof information from the document may vary from client to client. Thedocument format may indicate where, on a document, particular pieces ofdata may be located.

A document may be received. The document may be encrypted and signed.The engine 502 may verify the signature and decrypt the document. Theengine 502 may raise an event or command to the routing control unit506.

The routing control unit 506 determines document data based on thedocument and the document format (step 604). In various embodiments, thedocument format may indicate where on the document particular data canbe found, and the routing control unit 506 determines the document datafrom the document. In various embodiments, the document format indicatesa location in two dimensional space where various pieces of documentdata may be found on the document. In various embodiments, the documentformat indicates a word to be searched for, which is associated withvarious pieces of document data.

The document data may include a document type and subtype, a sender, areceiver, tracking information, and an item identifier associated withthe good, product, or service being ordered. The routing control unit506 may also validate relationships, apply strategies, store routing andtracking contexts, publish visibility, or perform pre-validationactions, such as duplicating the purchase order.

The routing control unit 506 generates a processed document based on thedocument data (step 606). The processed document may be the document tosend to the vendor 510. The routing control unit 506 determines missingdata from the document data, and supplements the processed document withadditional document data associated with the client (step 608). Theadditional document data may include billing data, shipping data,pricing data, routing data, communications channel data, and filenamedata. In various embodiments, supplementing the processed document withadditional document data may be applying rules to enrich the document.

Document security may be applied by the routing control unit 506 (step610). The document security may include encryption. The processeddocument is communicated to the vendor 510 (step 612).

In various embodiments, the client may be bound by using some standardof some form that may not be recognized by a vendor. The systeminterprets the form used by the client to communicate to the vendor thedocument data from the client.

In various embodiments, in response to the system receiving anon-standard form, the system overrides the standard form and uses thenon-standard form. Without the system, the non-standard form would berejected, and time and money would be spent to adjust the client'scommunication to normalize the data across all clients and vendors.

In various embodiments, various pieces of information may be misplacedon a document. For example, the sender and receiver may be misplaced ona purchase order. The system may analyze the purchase order, and basedon a combination of factors, may allow the modification of the endresult based on these heuristic rules.

In various embodiments, in response to no sender being identified, thesystem may determine the sender by tracing a connection by which thedocument was communicated. The system may determine whether an authenticconnection was used, or whether the file is encrypted and signed. In thecase the document is signed, the system may determine that whoeversigned the document is the sender.

The system provides a flexible and scalable business to businessdocument exchange for healthcare and other domains. FIG. 7 illustratesan exemplary architecture of the system. The system combines a set ofcloud and non-cloud friendly frameworks and algorithms to properlycomplete, align and enrich the business transaction. The system mayinclude one or more of a reconfigurable state machine, a communitytrading partner information hierarchical registry, a sender/receiver setof configurable strategies, a configurable extraction/validation engine,a business rules and transformation engine, a routing rules engine whichincludes flexible actions, a secure set of communication adapters, and abuilt-in audit and production like replay feature that helps improvequality.

The system separates the processing compute resources from theoperational compute resources in order to achieve real-time interactionsas well as to elastically scale. The system uses streaming whereapplicable for security and storage compute processing. A special schemeis used to tie the non-repudiation audit (track every actions taken)trail from the endpoint system all the way to the receiver, if deployedon both sides.

A special set of audit and quality features supports migration from adifferent platform onto the system. Business documents can be processedon both systems but remain live (delivered to vendor or client) on onesystem. The documents are automatically compared on both systems as partof an audit process to ensure no differences exist between the twoplatforms.

In various embodiments, a client healthcare provider (buyer of goods)sends a purchase order (or batch of them) to the system from theirinternal systems, destined to a vendor (seller of goods). The client istypically a facility of a larger organization, and the Ship To as wellas the Bill To address information can be different. The clients andvendors do not typically have a single source of truth to maintainproper information (e.g., Ship To address, Bill To address, CatalogueItems, Contracted Item Pricing). Conventionally, the client includes asupplier assigned account number which is used to uniquely identify thebuyer from the supplier perspective. Unique cases exist where some othercombination of buyer specific codes is included in the document instead.The system acts as the single source of truth registry for all partnersto properly determine the correct Ship To address, Bill To address, Iteminformation and pricing. This registry is used to adjust the purchaseorder accordingly and deliver the business document to the seller.Without these adjustments, the ordering process would fail. A similarprocess takes effect for clients returning business documents (e.g.,Purchase Order Acknowledgement, Advanced Ship Notice and Invoices).

In various embodiments, the system may be used in system-to-system dataintegration between a client system and applications, includingsynchronization of enriched data back into client systems from solutionssuch as catalogue and contracting applications. The system provides aflexible and scalable architecture to support high volume of real-timeinteractions for trading communities. The system includes a framework,which is a high throughput and latency-efficient event sub-system withconsumers/consumers/subscribers' components that minimizes the delay forstaged event driven architecture business to business platform. Theframework is designed to be robust, elastic and to support optimal loaddistribution among component instances running on the engines. A specialstructure is used to tie the events and information from the componentsthat performed work in response to these events.

Components receiving documents from remote systems process the receiveddocument and raise an event so that other components do the nextprocessing step. The process repeats until a correct business documentis delivered to its destination. The event system is the broker of theseevents. The event system manages to deliver the events to registeredcomponents in a loosely coupled fashion.

Leveraging the available features of the event subsystem, the triggersubsystem monitors configurable events and triggers processing logicembedded inside action components. Actions can raise alert messages tobe delivered by the alerting subsystem. A special framework is exposedto collect information for use to monitor documents and raise alerts inresponse to the trends of the exchanged documents violating certainconfigurable thresholds.

The system may include inbound engine and inbound adapters which can beintroduced to the platform in a dynamic fashion. There are two types ofinbound adapters: listener and puller. The listener component waits forconnections from remote systems and accepts secure documents. The pullerconnects to the remote system and retrieves the document into thesystem.

As illustrated in FIG. 8, the system includes a DREV (DocumentRecognition, Extraction and Validation) component to provide the abilityto dynamically recognize the metadata about a document via its contents.The components responsible for the recognition can be injected into thesystem at runtime. A file is received (802). Once the document has beenidentified (804), the document is quickly scanned for all pertinentinformation, again dynamically configured at runtime, and thisinformation is extracted into the routing context for further routing.

This event driven component also validates the document (806) forcontent and structure. If the document fails validation, the systeminitially attempts to find all applicable validation instructions (808)that would successfully validate the document and apply thoseinstructions (810). If that fails, the system may attempt, again basedon dynamic components, to format the document (812) so it passesvalidation. This validation methodology ensures the system can handleany type of document, regardless of the state of the document.

The secondary purpose of the DREV component is to do a full documentextraction (814), taking all information from a document and storing itin a common model that pertains to the business concept. This extractioninto the tracking context allows the document to be accessed by multipleapplications, irrespective of the format the information initially camein as. This process can also be reversed, in effect creating any type ofdocument from this tracking context.

Rules processing is triggered after the document is identified andinformation is extracted from the document. An event triggers the rulesengines that orchestrate the execution of actions based on the senderfollowed by actions based on the receiver. The actions are capable ofexecuting business logic including transformation of the document androuting the document to the proper outbound adapter for final delivery.

The business rules action can be engaged just like any other actionwithin the rules engine. The purpose of the business rules engine is tovalidate, enrich and/or correct information in the business documents.The logic can be applied on a specific sender and/or on receiver.

The outbound engines host outbound adapters which can be introduced tothe platform in a dynamic fashion. These adapters are referred to assenders since the purpose of them is to deliver the document to local orremote applications endpoints.

The visibility subsystem operates on the logical content of the businessdocuments (aka line items). It tracks the status on the documents andcorrelates the information to the return documents. For example, logicalinformation is tracked from the purchase orders and then correlates tothe information from purchase order acknowledgements, advanced shipnotice and correspondent invoice.

Production-Like Testing and Complex Business to Business Auditing System

The production-like testing and complex business to business auditingsystem includes a real-time verification feature that ensures that twoproduction systems will yield the same or similar final businesstransaction, yet only one system is selected to process the businessdocument. There may be millions of trading relations within an exchange,and a manual approach to migrate from an old system to a new system cancreate major disruptions in the trading community.

The system migrates the processing of documents from one system(typically an older platform/system being sunset) to the newer platformwithout causing disruption (or minimal disruption) to the clients andvendors. Without the system, the risk of misaligned orders may cause amajor business disruption raging from ordering the wrong goods orcompletely missing the orders.

The logic is applied to the trading relation (e.g., sender, receiver,business document type, or business document subtype) to perform theverification. The system verifies (in real-time/live in production mode)that the software functionality as well as the trading relationsconfiguration are functioning as expected. The system uses aconfigurable comparison algorithm to select the key elements to compare.

The feature allows the operator to control several key parameters, suchas which elements of the business transaction to compare, the number ofsuccessful audits before considering the verification successful, andthe desired action to trigger in response to the success criteria beingsatisfied. For example, the desired action may be to automatically movethe processing to the system or alert the operator to use it manually.

FIG. 9 illustrates a block diagram of the system. The system includes acloud-based production environment 902 and a cloud-based testingenvironment 904. Within the cloud-based production environment 902 is acloud-based production processing unit 906. Within the cloud-basedtesting environment 904 is a cloud-based testing processing unit 908.The cloud-based production environment 902 and the cloud-based testingenvironment 904 are in communication with a testing comparison unit 910,which is in communication with a display 912.

The system implements a scalable and efficient automated testing andquality assurance process for verifying a complex business to businessplatform. The system provides the ability to validate and ensure systemstability between code releases, as well as cross-team efficienciesaround updating partner configurations, new account creation and featurechanges. This is achieved by allowing the user to run a diverseselection of real data in a regulated fashion on a parallel andcontrolled environment while capturing performance statistics forparticipating components and then doing historical comparisons of theoutput.

The system provides production-like replay testing, which has thecapability of creating a set of millions of business documents from aproduction matching user defined criteria, provision a new parallel,controlled, isolated, on-demand and dispensable environment from scratchand execute the desired document set on it.

During the execution, various statistical information and metrics aregathered about the system cluster and its participating components toderive how the system is performing as a whole and by parts. This datacan then be compared with previous runs to find the deviations fromprevious releases. The verification may include execution time, status,state, mapping, document body data, metadata, system and component levelperformance statistics. Results can be compared from release to releaseto identify trends and help make proactive counter measures. Results fora release may be stored in terms of snapshots for future reference orcomparisons. This testing may assist in identifying bugs that affecteven one out of the millions of event flows.

The system insulates the business from major incidents that can causesignificant penalties as well as disruptions to the trading communities.The system migrates documents and verifies the accuracy of theinformation. In response to a new piece of software being released,where a particular document is stored and what functions may be appliedto it may be unpredictable. Many permutations and combinations may bepossible. The most secure way to verify the system is to retest with thesame configuration as before and replay in the same order that thedocument came in, and compare and to see if any of the software changesthat we made recently caused a drift in features or issues at all. Thesame production data is replayed. The final document is not communicatedoutside of the testing environment. Instead, it is contained within thetesting environment.

FIG. 10 illustrates a process 1000 of the production-like testing andcomplex business to business auditing system. A cloud-based productionenvironment 902 is constructed (step 1002) and a cloud-based testingenvironment 904 is constructed (step 1004). The cloud-based productionenvironment 902 may be an already existing, currently functioningenvironment.

A document is input to the original software in the cloud-basedproduction environment 902 (step 1006). The document is processed by thecloud-based production processing unit 906, executing the originalsoftware (step 1008). The document is input to the updated software inthe cloud-based testing environment 904 (step 1010). The document isprocessed by the cloud-based testing processing unit 908, executing theupdated software (step 1012).

The testing comparison unit 910 compares the initial software result andthe updated software result (step 1014). The testing comparison unit 910also compares the initial software process and the updated softwareprocess (step 1016). The respective software results may be the endproduct of the respective software, and the respective softwareprocesses may be the intermediate results and processes obtained andperformed to reach the respective end product of the respectivesoftware. The original software trail and the updated software trail arecompared.

The display 912 displays the comparison of the initial software resultand the updated software result and the comparison of the initialsoftware process and the updated software process (step 1018). Inresponse to any of the compared aspects between the original softwareand the updated software not partially or fully matching, the systemdetermines whether the mismatch is intended. In some situations, themismatch may be intended, as the updated software has addressed an erroror shortcoming in the original software. In response to the comparedaspects between the original software and the updated software matching,and a match is intended, it is determined that the updated software isverified.

In various embodiments, the same exact production data is replayed, withall the variation that may occur. The user may selectively focus onpurchase orders going from a particular client to a particular vendor.Thorough testing may be beneficial to avoid miscommunication andpossible penalties therefrom.

All of the processing is performed on the cloud, using cloud computing.FIGS. 11 and 12 illustrate the implementation of the system on acloud-based environment. In order to have a valid test, the productionenvironment is replicated. As the computing is performed on the cloud,in various embodiments, the system may not have any associated hardware.Everything is virtualized on the cloud. The system may first build outan entire production like environment in the cloud. The number ofservers in production is replicated, and all of them are put on thecloud. Tasks are provisioned, and a production environment is created.

Production for one week or production for one day may be replayed, forexample. What to replay may be selected and the system may replay, forexample, the entire business process, including the purchase coming in,invoice out, advance ship notice, and purchase acknowledgement. Anenvironment is provisioned on the cloud, dynamically. The environment isset up one machine at a time. The environment reaches production, pullsall of the metadata that triggers the replay, and performs the replay.Once the replay is done, it compares the result from the new testingenvironment to the result that is in production, and displays anydiscrepancies.

For example, for a particular document that was previously processed,the entire processing of the document and the triggered sub-systems maybe replayed on the updated software. When the document is a purchaseorder, the events triggered by the receiving of the purchase order(e.g., verifying signature, decrypting the document, identifyingformatting of the purchase order, enriching the purchase order) may alsobe seen and recorded. When the purchase order replay is complete, theresults from the updated software are compared to the results from whenthe particular document was actually previously processed, to determinewhether there are changes, and if so, whether the changes were intended.

There may be two kinds of databases, a configuration database and theruntime database. The configuration between the testing environment andthe production environment must be identical. Therefore, theconfiguration is saved to the configuration database. Once theconfiguration is copied, the purchase order is obtained, the line itemsare fix up, the line items are aligned to the suppliers catalog totranslate for different terms for each line item. Depending on therules, the order may be split into multiple orders. Finally, it is sentit to the vendor using a specific protocol. However, as this is a test,instead of sending it to the endpoint of the vendor, it is sent to aninternal mock-up application.

A graphical user interface dashboard may be provided that displays howmany failures occurred, how many success occurred, and the user mayfurther analyze the data to determine what failed and what did not.

Performing the testing on the cloud enables a significant cost savings.An entire testing environment may be provisioned and then destroyed,without having to purchase hardware for the test.

Conventionally, testing is done with use cases that may not includeexactly what is in production. This may result in a test not setting upcorrectly and having the right data in the files. Instead, the systemreplays the production, but the end execution is not communicated to avendor.

The system provides the ability to do a final verification in productionand migrate in a controlled manner few relations at a time. The systemalso provides a rollback option which allows for a fast undo. The systemis also capable of predicting system performance based on variation onload, introduction of a component, onboardings, and test migrationstrategies and their effects. The system provides the ability to replayan entire historical timeline from the production environment and verifythat the business transactions remain consistent. The system alsoprovides a way of identifying bottlenecks in the system, optimizing thequality of the system.

In some situations, in response to a system being merged with anothersystem, a particular document may be processed in the first system andthe second system. The system may provide for an audit flow to test thedocument in the first system and the second system. The flow may begoverned by the sender, the receiver, the document type. The systemallows the controlled consolidation from an old system to a new system.

Any of the systems described herein may communicate with a smartphone,the internet and/or social networking websites. Any communication,transmission and/or channel discussed herein may include any system ormethod for delivering content (e.g. data, information, metadata, etc),and/or the content itself. The content may be presented in any form ormedium, and in various embodiments, the content may be deliveredelectronically and/or capable of being presented electronically. Forexample, a channel may comprise a website or device (e.g., Facebook,YOUTUBE®, APPLE®TV®, PANDORA®, XBOX®, SONY® PLAYSTATION®), a uniformresource locator (“URL”), a document (e.g., a MICROSOFT® Word® document,a MICROSOFT® Excel® document, an ADOBE® .pdf document, etc.), an“ebook,” an “emagazine,” an application or microapplication (asdescribed herein), an SMS or other type of text message, an email,facebook, twitter, MMS and/or other type of communication technology. Invarious embodiments, a channel may be hosted or provided by a datapartner. In various embodiments, the distribution channel may comprisein response to of a merchant website, a social media website, affiliateor partner websites, an external vendor, a mobile device communication,social media network and/or location based service. Distributionchannels may include in response to of a merchant website, a socialmedia site, affiliate or partner websites, an external vendor, or amobile device communication. Examples of social media sites includeFACEBOOK®, FOURSQUARE®, TWITTER®, MYSPACE®, LINKEDIN®, and the like.Examples of affiliate or partner websites include AMERICAN EXPRESS®,GROUPON®, LIVINGSOCIAL®, and the like. Moreover, examples of mobiledevice communications include texting, email, and mobile applicationsfor smartphones.

In various embodiments, components, modules, and/or engines of thesystem may be implemented as micro-applications or micro-apps.Micro-apps are typically deployed in the context of a mobile operatingsystem, including for example, a WINDOWS® mobile operating system, anANDROID® Operating System, APPLE® IOS®, a BLACKBERRY® operating systemand the like. The micro-app may be configured to leverage the resourcesof the larger operating system and associated hardware via a set ofpredetermined rules which govern the operations of various operatingsystems and hardware resources. For example, where a micro-app desiresto communicate with a device or network other than the mobile device ormobile operating system, the micro-app may leverage the communicationprotocol of the operating system and associated device hardware underthe predetermined rules of the mobile operating system. Moreover, wherethe micro-app desires an input from a user, the micro-app may beconfigured to request a response from the operating system whichmonitors various hardware components and then communicates a detectedinput from the hardware to the micro-app.

The system may communicate with any network using any datacommunications protocol. As used herein, the term “network” includes anycloud, cloud computing system or electronic communications system ormethod which incorporates hardware and/or software components.Communication among the parties may be accomplished through any suitablecommunication channels using any communications protocol, such as, forexample, a telephone network, an extranet, an intranet, Internet, pointof interaction device (point of sale device, personal digital assistant(e.g., IPHONE®, BLACKBERRY®), cellular phone, kiosk, etc.), onlinecommunications, satellite communications, off-line communications,wireless communications, transponder communications, local area network(LAN), wide area network (WAN), virtual private network (VPN), networkedor linked devices, keyboard, mouse and/or any suitable communication ordata input modality. Moreover, although the system is frequentlydescribed herein as being implemented with TCP/IP communicationsprotocols, the system may also be implemented using IPX, APPLE®talk,IP-6, NetBIOS®, OSI, any tunneling protocol (e.g. IPsec, SSH), or anynumber of existing or future protocols. If the network is in the natureof a public network, such as the Internet, it may be advantageous topresume the network to be insecure and open to eavesdroppers. Specificinformation related to the protocols, standards, and applicationsoftware utilized in connection with the Internet is generally known tothose skilled in the art and, as such, need not be detailed herein. See,for example, DILIP NAIK, INTERNET STANDARDS AND PROTOCOLS (1998); JAVA®2 COMPLETE, various authors, (Sybex 1999); DEBORAH RAY AND ERIC RAY,MASTERING HTML 4.0 (1997); and LOSHIN, TCP/IP CLEARLY EXPLAINED (1997)and DAVID GOURLEY AND BRIAN TOTTY, HTTP, THE DEFINITIVE GUIDE (2002),the contents of which are hereby incorporated by reference.

“Cloud” or “Cloud computing” includes a model for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, servers, storage, applications, and services)that can be rapidly provisioned and released with minimal managementeffort or service provider interaction. Cloud computing may includelocation-independent computing, whereby shared servers provideresources, software, and data to computers and other devices on demand.For more information regarding cloud computing, see the NIST's (NationalInstitute of Standards and Technology) definition of cloud computing athttp://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf (lastvisited June 2012), which is hereby incorporated by reference in itsentirety.

The computers discussed herein may provide a suitable website or otherInternet-based graphical user interface which is accessible by users. Inone embodiment, the MICROSOFT® INTERNET INFORMATION SERVICES® (IIS),MICROSOFT® Transaction Server (MTS), and MICROSOFT® SQL Server, are usedin conjunction with the MICROSOFT® operating system, MICROSOFT® NT webserver software, a MICROSOFT® SQL Server database system, and aMICROSOFT® Commerce Server. Additionally, components such as Access orMICROSOFT® SQL Server, ORACLE®, Sybase, Informix MySQL, Interbase, etc.,may be used to provide an Active Data Object (ADO) compliant databasemanagement system. In one embodiment, the Apache web server is used inconjunction with a Linux operating system, a MySQL database, and thePerl, PHP, and/or Python programming languages.

Any of the communications, inputs, storage, databases or displaysdiscussed herein may be facilitated through a website having web pages.The term “web page” as it is used herein is not meant to limit the typeof documents and applications that might be used to interact with theuser. For example, a typical website might include, in addition tostandard HTML documents, various forms, JAVA® APPLE®ts, JAVASCRIPT,active server pages (ASP), common gateway interface scripts (CGI),extensible markup language (XML), dynamic HTML, cascading style sheets(CSS), AJAX (Asynchronous JAVASCRIPT And XML), helper applications,plug-ins, and the like. A server may include a web service that receivesa request from a web server, the request including a URL and an IPaddress (123.56.789.234). The web server retrieves the appropriate webpages and sends the data or applications for the web pages to the IPaddress. Web services are applications that are capable of interactingwith other applications over a communications means, such as theinternet. Web services are typically based on standards or protocolssuch as XML, SOAP, AJAX, WSDL and UDDI. Web services methods are wellknown in the art, and are covered in many standard texts. See, e.g.,ALEX NGHIEM, IT WEB SERVICES: A ROADMAP FOR THE ENTERPRISE (2003),hereby incorporated by reference.

The system may also create, maintain and/or supplement a user profile. A“user profile” or “user profile data” may comprise any information ordata about a consumer that describes an attribute associated with theconsumer (e.g., a preference, an interest, demographic information,personally identifying information, and the like).

Benefits and other advantages have been described herein with regard tospecific embodiments. Furthermore, the connecting lines shown in thevarious figures contained herein are intended to represent exemplaryfunctional relationships and/or physical couplings between the variouselements. It should be noted that many alternative or additionalfunctional relationships or physical connections may be present in apractical system. However, the benefits, advantages, and any elementsthat may cause any benefit or advantage td occur or become morepronounced are not to be construed as critical, required, or essentialfeatures or elements of the disclosure. The scope of the disclosure isaccordingly to be limited by nothing other than the appended claims, inwhich reference to an element in the singular is not intended to mean“one and only one” unless explicitly so stated, but rather “one ormore.” Moreover, where a phrase similar to “in response to of A, B, orC” is used in the claims, it is intended that the phrase be interpretedto mean that A alone may be present in an embodiment, B alone may bepresent in an embodiment, C alone may be present in an embodiment, orthat any combination of the elements A, B and C may be present in asingle embodiment; for example, A and B, A and C, B and C, or A and Band C.

Systems, methods and apparatus are provided herein. In the detaileddescription herein, references to “various embodiments”, “oneembodiment”, “an embodiment”, “an example embodiment”, etc., indicatethat the embodiment described may include a particular feature,structure, or characteristic, but every embodiment may not necessarilyinclude the particular feature, structure, or characteristic. Moreover,such phrases are not necessarily referring to the same embodiment.Further, when a particular feature, structure, or characteristic isdescribed in connection with an embodiment, it is submitted that it iswithin the knowledge of one skilled in the art to affect such feature,structure, or characteristic in connection with other embodimentswhether or not explicitly described. After reading the description, itwill be apparent to one skilled in the relevant art(s) how to implementthe disclosure in alternative embodiments.

System program instructions and/or controller instructions may be loadedonto a tangible, non-transitory, computer-readable medium (also referredto herein as a tangible, non-transitory, memory) having instructionsstored thereon that, in response to execution by a controller, cause thecontroller to perform various operations. The term “non-transitory” isto be understood to remove only propagating transitory signals per sefrom the claim scope and does not relinquish rights to all standardcomputer-readable media that are not only propagating transitory signalsper se. Stated another way, the meaning of the term “non-transitorycomputer-readable medium” and “non-transitory computer-readable storagemedium” should be construed to exclude only those types of transitorycomputer-readable media which were found in In Re Nuijten to falloutside the scope of patentable subject matter under 35 U.S.C. § 101.

Furthermore, no element, component, or method step in the presentdisclosure is intended to be dedicated to the public regardless ofwhether the element, component, or method step is explicitly recited inthe claims. No claim element herein is to be construed under theprovisions of 35 U.S.C. 112(f), unless the element is expressly recitedusing the phrase “means for.” As used herein, the terms “comprises”,“comprising”, or any other variation thereof, are intended to cover anon-exclusive inclusion, such that a process, method, article, orapparatus that comprises a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus.

What is claimed is:
 1. A method of testing updated software comprising:constructing a cloud-based production environment; constructing acloud-based testing environment; executing an initial software in thecloud-based production environment by inputting a document; determiningan initial software result and an initial software process based onexecuting the initial software with the document; executing the updatedsoftware in the cloud-based testing environment by inputting thedocument; determining an updated software result and an updated softwareprocess based on executing the updated software with the document;comparing the initial software result and the updated software result;and comparing the initial software process and the updated softwareprocess.
 2. The method of claim 1, wherein the document is associatedwith a client and a vendor.
 3. The method of claim 2, wherein thedocument comprises a plurality of documents associated with the clientand the vendor.
 4. The method of claim 3, wherein the plurality ofdocuments associated with the client and the vendor are furtherassociated with a time range.
 5. The method of claim 1, wherein thedocument comprises a plurality of documents having a document type. 6.The method of claim 1, wherein a configuration of the cloud-basedproduction environment is identical to a configuration of thecloud-based testing environment.
 7. The method of claim 1, furthercomprising displaying, by a display, the comparison of the initialsoftware result and the updated software result and the comparison ofthe initial software process and the updated software process.
 8. Themethod of claim 1, wherein the document is a document being processed ina non-testing situation by an end user.
 9. A system of testing updatedsoftware comprising: a cloud-based production environment; a cloud-basedtesting environment; a cloud-based production processing unit configuredto: receive a document, execute an initial software in the cloud-basedproduction environment by inputting the document in the cloud-basedproduction environment, and determine an initial software result and aninitial software process based the document; a cloud-based testingprocessing unit configured to: receive the document, execute an updatedsoftware in the cloud-based testing environment by inputting thedocument in the cloud-based testing environment, and determine anupdated software result and an updated software process based on thedocument; and a testing comparison unit configured to compare theinitial software result and the updated software result, and compare theinitial software process and the updated software process.
 10. Thesystem of claim 9, wherein the document is associated with a client anda vendor.
 11. The system of claim 10, wherein the document comprises aplurality of documents associated with the client and the vendor. 12.The system of claim 11, wherein the plurality of documents associatedwith the client and the vendor are further associated with a time range.13. The system of claim 9, wherein the document comprises a plurality ofdocuments having a document type.
 14. The system of claim 9, wherein aconfiguration of the cloud-based production environment is identical toa configuration of the cloud-based testing environment.
 15. The systemof claim 9, further comprising a display configured to display thecomparison of the initial software result and the updated softwareresult and the comparison of the initial software process and theupdated software process.
 16. The system of claim 9, wherein thedocument is a document being processed in a non-testing situation by anend user.
 17. A cloud-based testing system for testing updated softwarecomprising: a cloud-based testing processing unit configured to: receivean in response to document, execute an updated software in thecloud-based testing environment by inputting the document in thecloud-based testing environment, and determine an updated softwareresult and an updated software process based on the document; and atesting comparison unit configured to: compare the updated softwareresult with an initial software result received from a cloud-basedproduction processing unit based on executing an initial software in acloud-based production environment by inputting the document, andcompare the updated software process with an initial software processreceived from the cloud-based production processing unit based onexecuting the initial software in the cloud-based production environmentby inputting the document.
 18. The cloud-based testing system of claim17, wherein the document is associated with a client and a vendor. 19.The cloud-based testing system of claim 18, wherein the documentcomprises a plurality of documents associated with the client and thevendor.
 20. The cloud-based testing system of claim 19, wherein theplurality of documents associated with the client and the vendor arefurther associated with a time range.